Skip to main content

Reply to this post | See parent post | Go Back
View Post [edit]

Poster: glenn Date: May 7, 2003 1:29pm
Forum: etree Subject: Re: Putting email addresses in the details pages

there do exist user-agents which ignore the robots.txt exclusion file, so it is safe to assume that all portions of this site can be crawled by malicious (email-address-harvesting) user-agents.
A better place to put seeder's email addresses, in my opinion, is in the info.txt file included with the seed.

Reply to this post
Reply [edit]

Poster: Jonathan Aizen Date: May 7, 2003 2:24pm
Forum: etree Subject: Re: Putting email addresses in the details pages

Again, the text files are crawled, so that won't help...